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A SHARPER DISCREPANCY MEASURE FOR 
POST-ELECTION AUDITS 

By Philip B. Stark 

University of California — Berkeley 

Post-election audits use the discrepancy between machine counts 
and a hand tally of votes in a random sample of precincts to infer 
whether error affected the electoral outcome. The maximum relative 
overstatement of pairwise margins (MRO) quantifies that discrep- 
ancy. The electoral outcome a full hand tally shows must agree with 
the apparent outcome if the MRO is less than 1. This condition is 
sharper than previous ones when there are more than two candidates 
or when voters may vote for more than one candidate. For the 2006 
U.S. Senate race in Minnesota, a test using MRO gives a P- value of 
4.05% for the hypothesis that a full hand tally would find a different 
winner, less than half the value Stark [Ann. Appl. Statist. 2 (2008) 
550-581] finds. 

1. Maximum relative overstatement of pairwise margins. For a candi- 
date other than an apparent winner to be a real winner of an election, 
error that hurts that candidate or helps an apparent winner must exceed 
that apparent winner's margin of victory over that candidate. The max- 
imum relative overstatement of pairwise margins (MRO) takes that into 
account; previous measures compare errors with the margin of victory over 
the runner-up alone. 

Consider a contest with K candidates, 1, . . . ,K , and N precincts, 1, . . . ,N. 
Each voter may vote for up to / candidates. The / candidates who appar- 
ently won are those in KL W . Those who apparently lost are in /Q. The appar- 
ent vote for candidate k in precinct p is Vk p - The apparent vote for candidate 
k is Vk = J2p=i v kp- The apparent margin of candidate w over candidate £ is 
V w £ = V W — Vi. For w € 1C W and t G /Q, V w t > 0: the apparent winners are the 
/ candidates with strictly positive apparent margins over the other K — f. 

Actual, as a modifier of vote, margin, winner or electoral outcome, means 
what a full hand tally would show. The actual vote for candidate k in 
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precinct p is a^ p . The actual vote for candidate k is A}~ = J2p=i a kp- The 
actual margin of candidate w over candidate £ is A w £ = A w — Ag. The ap- 
parent winners are the actual winners if 

(1) cl F? cr ^ > °- 

Define 

/ 9 v _ (t'top — ?%>) — (a wp — ai p ) 

For the apparent and actual electoral outcomes to differ, there must exist 
w G 3C W and £ € /Q for which X^=i e p«^ ^ 1- The maximum relative over- 
statement of pairwise margins (MRO) in precinct p is 

(3) e p = max e pw£ . 



Now 



N N N 



(4) max ^ < £ max e p ^ = ^ e p . 

p=l p=l p=l 

The sum on the right is the maximum relative overstatement of pairwise 
margins (MRO). If the apparent and actual electoral outcomes differ, J2 P =i e p — 
1. When K = 2 and / = 1, this is equivalent to the condition Stark (2008) 
tests. But for K > 2 or / > 1, this condition can be much sharper. 

Suppose the number of valid ballots cast in precinct p is at most b p . 
Clearly, a wp > and ai p < b p . Hence, e pw i < (v wp — V£ p + b p )/V w £, and so 

(5) e p < max + 



Let {w p (-)}p = i be monotonic functions. Stark's (2008) method can test the 



hypothesis that J2p=i e p > 1 given the constraint e p < u p using the maximum 
observed value of w p (e p ) as the test statistic: substitute M = 1 and the 



definitions of u and e p given here. 



2. The 2006 U.S. Senate race in Minnesota. Table 1 lists the vote totals 
for the 2006 U.S. Senate race in Minnesota. The apparent winner was Amy 
Klobuchar. 1 Minnesota elections law S.F. 2743 (2006) requires that counties 
with fewer than 50,000 registered voters audit at least two precincts chosen 
at random; that counties with between 50,000 and 100,000 registered voters 



See www. sos . state .mn.us/docs/2006_General_Results .XLS, 

electionresults.sos.state.mn. us/2006 1107/ElecRslts.asp?M=S&Races=0102, and 
www. sos . state .mn. us/home/ index . asp?page=544 . 
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Table 1 

Summary of 2006 U.S. Senate race in Minnesota 



Voters Undervotes Fitzgerald Kennedy Klobuchar Cavlan Powers Write-ins 

& invalid (Indep) (Repub) (Democ/Farm/ (Green) (Constit) 
ballots Labor) 

2,217,818 15,099 71,194 835,653 1,278,849 10,714 5,408 901 

V wt N/A 1,207,655 443,196 N/A 1,268,135 1,273,441 1,277,948 



audit at least three; and that counties with more than 100,000 registered 
voters audit at least four. At least one precinct audited in each county 
must have 150 or more votes. Minnesota has 4,123 precincts in 87 counties. 
202 precincts were audited after the 2006 election. Several counties audited 
more than the legal minimum. 

Following Stark (2008), we pool Cavlan, Powers, and the write-in can- 
didates to form a pseudo-candidate who apparently lost to Klobuchar by 
1,261,773 votes. Thus, K = 4, / = 1 and N = 4,123. The maximum value of 
Up is 0.0097. The maximum observed value of e p is 4.5 x 10~ 6 . If Klobuchar 
actually lost, the MRO in at least 166 precincts must be larger than any in 
the sample. In contrast, for the measure of margin overstatement Stark (2008) 
uses, only about 130 precincts would need to have values exceeding any in 
the sample. 2 Thus, it is easier to confirm that the apparent and actual out- 
comes agree using the MRO. 

We calculate a conservative P- value for the hypothesis that Klobuchar ac- 
tually lost by pretending that the sample was drawn with replacement from 
all 4,123 precincts, but that only 78 precincts were drawn, as if the popula- 
tion were sampled using the minimum sampling fraction among counties. 3 
For weight functions w p (x) = x, the P-value is the maximum chance that 
78 precincts chosen at random with replacement would have e p < 0.0097 if, 
among all 4,123 precincts, there were at least 166 with e p > 0.0097. That 
value is ( 412 4 3 1 ~ 3 166 ) 78 = 4.05%, roughly half the conservative P-value of 8.22% 
Stark (2008) finds. 

If 202 precincts were drawn as a simple random sample and the same 
discrepancies were observed, the P-value would be about 0.02% using the 
MRO. In contrast, Stark (2008) finds a corresponding P-value of about 
0.13%. 

3. Conclusion. The MRO yields a sharper necessary condition for the 
apparent electoral outcome to differ from the outcome a full hand tally 



2 See Table 5 of Stark (2008). 
3 See Section 4.2.1 of Stark (2008). 
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would show than previous measures of the discrepancy between machine 
and hand counts do. An a priori bound for the MRO in a precinct can be 
derived from a bound on the number of valid ballots in that precinct. The 
testing framework Stark (2008) develops works with MRO if the definitions 
of M, u and e are revised, and yields a more powerful test. 
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